Usability Analysis of Concept Maps as Knowledge Discovery Tools

The potential application of concept maps in knowledge discovery is examined using a framework comprising usability, learnability, and task & technology fit. An online survey and follow-up interviews indicated that teachers and administrators at the Ministry of Education in Singapore were comfortable in using concept maps. They expressed that concept maps should to be integrated with search engine tools. Usability analysis suggested that enhancements in functional design of concept maps, improvement in quality of contents, and regular updating of resources were important to attract mass usage of concept maps. INTRODUCTION Concept maps are one of the most notable methods for representing relationships between concepts. Using this premise as a basis for further investigations, many researchers have begun to analyse various applications of concept maps, including organising and presenting information to aid knowledge discovery. Concept maps have their roots in education. The idea was grounded in the sound learning theories of Ausubel (Novak & Gowin, 1984) who posited the importance of prior knowledge in learning new information. The fundamental idea is assimilation theory where learning takes place by the assimilation of new concepts and propositions into existing concepts and prepositional frameworks held by the learner. Concept maps are tools for organizing and representing knowledge that include concepts; usually enclosed in circles or boxes of some type; and relationships between concepts or propositions; indicated by a connecting line between two concepts. Words on the line specify the relationship between the two concepts. The basic graphical elements are nodes and links, which are organised using common patterns, such as branches, arrows, groups, notes, lists and so on. Nodes (points, vertices, icons, and so on) represent the concepts, and links (lines or arcs) represent the relationships between concepts. Figure 1 portrays an example of a simple concept map. Concept maps are representations of a set of concepts and their relationships and concept mapping is a technique for representing knowledge or information in graphical form by listing relevant concepts and then drawing lines between them to represent their interrelationships. When two or more concepts are connected, a meaningful statement or proposition is formed. Propositions are statements about some objects or events in the universe, either occurring naturally or constructed (Cañas et al. 2003). They are also called semantic units, or units of meaning. Typically a concept is expressed using one or a few words with labelled links. These links are lines or curves with phrases that connect concepts to form relationships. A linking phrase is used to join concepts to form a meaningful proposition, which is a basic unit of knowledge according to the theory of meaningful learning. Nodes in a map do not carry any information. Arrowheads in the links specify directions which determine the logical connection of the relationships. Curved lines can be used when straight lines cannot be used to link the nodes, thus providing the flexibility to avoid rearrangement. Another important element of concept maps is Cross-links. Cross-links make explicit relationships between or among concepts in different domains within the concept map. Much research on users’ perceptions of concept maps has been done in the context of the tool’s application in education. Santhanam and Dawson (1998) investigated the effects of concept mapping to students taking subjects in genetics. Uzuntiryaki and Geban (2002) examined concept mapping instruction in the science subject by comparing it to traditional instruction. Potelle and Rouet (2003) studied the effects of concept maps on low and high knowledge students. Figure 1. An example of a concept map 866 2007 IRMA International Conference Copyright © 2007, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. Research on users’ perceptions of concept maps in the knowledge management field is not very prevalent. Much of it deals with knowledge elicitation and knowledge construction. Ford, et al. (1996) described a nuclear cardiology expert system called NUCES in which concept maps were used for knowledge elicitation and navigation. Leaker et al. (2003) described an approach to support experts as they built their own knowledge models of a domain. Mularz and Lyell (2004) explored the integration of concept maps and semantic web technologies for the capture, visualization and navigation of knowledge in support of the lifecycle of knowledge management. This study was aimed at investigating the potential application of concept maps in knowledge discovery. Teachers’ professional development was chosen as the central topic for developing a prototype concept map to be used for review. Relevant concepts were derived from selected information repositories and reference questions received by READ@TN (Teachers Network’s information resource centre at the Ministry of Education in Singapore). The prototype concept map covered a set of terms and their relationships, which also took references from Educause taxonomy (http://www.educause.edu/Browse/647), ERIC Thesaurus (http://eric.ed.gov/) & Library of Congress Subject Headings (LCSH). Relevant documents were categorised and linked to the established concepts and their associations. User feedback on the usefulness was of the concept map was sought from a group of teachers. This domain-specific concept map was implemented to function as a finding aid. Figure 2. Concept map on teachers’ professional development Managing Worldwide Operations & Communications with Information Technology 867 Copyright © 2007, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited. A framework comprising usability, learnability, usability, and task & technology fit was used for review of concept maps. Based on this framework, several parameters were identified to perform usability analysis by two main groups of stakeholders: teachers and administrators. It served as a map of the knowledge for a community-of-interest in the area of continuing professional development for teachers. DATA COLLECTION Original concepts for the map were derived from the writings of Diaz-Maggioli (2004) and Zmuda (2004). These concepts were subsequently presented to a team working on teachers’ professional development for comments and amendments. CmapTools, (an open source tool by the Institute for Human Machine Cognition (IHMC), University of West Florida) was used to construct the concept map. It allowed easy publishing of knowledge models in concept map servers and enabled concept maps to be linked to related concept maps and to other types of media. Linking phrases were added to the key concepts on the preliminary concept map to express the relationships among concepts. This was followed by adding crosslinks to illustrate the inter-relationships between concepts in different areas or sub-domains on the map. A total of 157 documents housed by READ@TN were linked to the relevant concepts. These included mostly documents in Microsoft Word and PDF format. In-house web pages and quality information from external websites were also linked to the concept map. Most concepts had one or two documents attached to them to give the respondents a flavour of what they could expect on the concept maps. The concept map was subjected to several rounds of reviews by senior education officers. Changes to the structure and contents were made prior to implementing the prototype on a hosting server (http://www.100free.com) at http://leeyf.100free.com. The availability of concept map on the Internet provided ease of access for participants. They could experience the working of the concept map at their convenience. On the basis of their experience they subsequently responded to a questionnaire survey (which was also web-based). Figure 2 show the final concept map constructed. Thirty education officers and five administrators participated in the online usability survey. The participants were invited to view the concept map after a demo was arranged to introduce the concept map to them. Eighteen participants signed up for the follow-up interview (three senior administrators, nine senior education officers and six junior education officers). FINDINGS Usability Three parameters including ease of use, effect, and satisfaction were used to review the usability aspects. Feedback from participants on these parameters is reported in the following section. Ease of Use Ease of use of the concept map encompasses the notion of simplicity and accessibility of finding and discovering information. Seventy-seven % participants reported that the concept map presented to them was easy to use. However, a fair portion of the respondents felt that, when the map was first presented to them, they were overwhelmed by the concept map and had difficulty understanding and using the map. Twenty-nine % indicated that navigating the concept map was easy. All respondents were able to follow the logic of the navigation. Most participants were able to appreciate the navigational logic of the concept map. A large majority was able to easily navigate the concept maps for relevant information. Sixty-eight per cent participants supported the proposition that concept maps improve the information searching output; while 14% did not agree that concept maps added value to the information searching experience. Despite the differences in opinion, all participants acknowledged concept map as a useful tool to aid knowledge discovery. They expressed that concept maps could provide guidance and direction to the information search process. Satisfaction Satisfaction is derived when a user is able to perform his/her information search task successfully. About one-third participants strongly supported the acceptance of concept maps while almost two-third expressed continuing interest to use concept maps in the future. When asked which group of Internet users could effectively use concept maps to search for information, 69% felt that concept maps would be accepted in all strata of Internet users; 71% felt that concept maps could only be accepted by novice Internet users while an astonishing 94% agreed that seasoned Internet users were capable to effectively use this finding aid. Forty-nine % felt comfortable using the concept maps. A further 31% were confident; only 17% experienced some stress while using concept maps to find information. The findings revealed a moderately high level of satisfaction and acceptance to the use of concept map in the context of teachers’ professional development. The fact that participants had indicated strongly their desire to use concept maps shows a high level of confidence within them in this information search tool. Effect The effect of concept maps is tied to performance and outcome of the information searching exercise. There is a good spread of responses on the effectiveness of concept maps compared with other online search tools such as Internet search engines, Internet directories, library online catalogue. Forty-two % participants maintained that concept maps were more effective while 32% felt otherwise. More than three-quarter (77%) either agreed or strongly agreed that their overall experience with concept maps was successful and positive. All respondents felt that concept maps were able to guide them in their information search. They expressed full confidence in concept maps reflecting that concept maps provide a sense of purpose in finding information. However, statistical analysis suggested that concept map alone was not able to replace the role of other search tools like Internet search engines (e.g. Google, MSN Search) and search directories (e.g. Yahoo Directory), as less than half believed it could do so. Learnability Learnability refers to the ease of learning the system’s functionality and gaining proficiency to complete the search for information. Our analysis attempted to review whether the concept map as an information search system encouraged holistic understanding and supported serendipitous discovery. We also reviewed if the concept map was able to enhance a broader understanding of concerning concepts surrounding each topic. When the concept map was developed, we expected that with the inter-relationships between various sub-topics (concepts) teachers would be able to gain breadth in their subject of interest. In the review, we wanted to Figure 3. Learnability aspects of use of concept maps Strongly Agree, 34% Strongly Agree, 17% Strongly Agree, 43% Strongly Agree, 34% Agree, 60% Agree, 54% Agree, 54% Agree, 60% No Idea, 3% No Idea, 17% No Idea, 0% No Idea, 6% Disagree, 3% Disagree, 11% Disagree, 3% Disagree, 0% Strongly Disagree, 0% Strongly Disagree, 0% Strongly Disagree, 0% Strongly Disagree, 0% 0% 10% 20% 30% 40% 50% 60% 70% Using concept maps, I